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The  Honorable  Les  Aspin 
Chairman,  Committee  on  Armed  Services 
House  of  Representatives 

Dear  Mr.  Chairman: 

As  you  requested,  we  have  reviewed  the  strategy  for  acquiring  the 
Short-Range  Unmanned  Aerial  Vehicle  (iav)  system.  Because  of  your 
Committee’s  concern  about  the  lack  of  success  on  past  i  av  programs  and 
the  need  to  assure  that  related  acquisition  weaknesses  do  not  recur,  we 
compared  the  Short-Range  i  av  acquisition  strategy  to  that  followed  in 
acquiring  the  Pioneer  i  av.  a  previously  procured  system  which  encoun¬ 
tered  problems  when  deployed  with  the  Navy's  operational  forces. 


Background 


The  Short-Range  i  av  program,  expected  to  cost  $1.5  billion,  w^as  initi¬ 
ated  as  a  joint-service  effort  in  response  to  congressional  concern  about 
the  proliferation  of  i  av  programs  and  the  need  to  acquire  i  av  systems 
that  could  meet  the  requirements  of  more  than  one  service.  In  this 
regard,  Congress  refused  to  authorize  fiscal  year  1988  funding  for  pro¬ 
curement  of  separate  Army  and  Navy  Short-Range  t'AVs  and  provided 
funds  only  for  a  joint  program.  Accordingly,  the  Department  of  Defense 
( non)  is  acquiring  the  Short-Range  iav  to  meet  the  needs  of  the  Army, 
Navy,  and  Marine  Corps. 

i ton  plans  to  begin  low  rate  production  of  the  Short-Range  system  in 
fiscal  year  1992  with  the  procurement  of  8  of  a  total  program  quantity 
of  53  systems.  Each  system  to  be  procured  initially  is  to  include  8  air 
vehicles,  and  the  <14  vehicles  are  to  be  deployed  in  1994. 


Results  in  Brief 


non's  acquisition  strategy  for  the  Short-Range  i  av  is  to  perform  opera¬ 
tional  testing  before  beginning  low  rate  production.  Despite  the  impor¬ 
tance  of  realistic  operational  testing,  however,  the  Short-Range  i  av 
acquisition  strategy  provides  for  testing  in  an  environment  not  represen¬ 
tative  of  where  the  system  is  supposed  to  be  deployed.  This  strategy 
places  non  at  risk  of  becoming  committed  to  the  production  of  a  system 
based  on  test  results  that  may  not  be  a  valid  indicator  of  the  system's 
capability.  The  strategy  is  similar  to  Pioneer's  strategy  that  resulted  in 
spending  $1(>()  million  for  a  system  that  still  does  not  meet  performance 
requirements. 
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dod's  acquisition  policies  require  that  operational  testing  be  held  in  a 
realistic  environment  representing  combat  conditions  to  the  extent  prac¬ 
tical.  Nevertheless,  dod  plans  to  limit  the  Short-Range  system's 
preproduction  operational  testing  to  areas  that  do  not  provide  environ¬ 
ments  typical  of  where  current  deployment  plans  indicate  the  system 
could  be  used  in  combat.  Most  of  this  preproduction  testing  will  be  lim¬ 
ited  to  a  desert  environment.  We  recognize  that  if  the  current  Middle 
East  situation  continues  until  1994,  when  the  system  is  to  be  deployed, 
the  Short-Range  system  deployment  plans  could  be  changed.  However, 
the  system's  currently  required  capability  of  locating  targets  under 
diverse  conditions  in  multiple  environments  is  not  likely  to  be  demon¬ 
strated  before  dod  becomes  committed  to  its  production. 

dod  also  plans  to  begin  full-rate  production  of  the  Short-Range  system 
before  verifying  that  it  can  be  modified  to  meet  Navy  requirements. 
Although  dod  plans  to  conduct  operational  testing  before  full-rate  pro¬ 
duction,  the  system  to  be  tested  will  not  incorporate  those  features  nec¬ 
essary  to  operate  in  a  naval  environment.  If  subsequent  testing  of  the 
Navy  variant  were  to  show  the  system  to  be  unsuitable  for  naval  use, 
dod  would  then  be  fully  committed  to  a  system  not  meeting  congres¬ 
sional  intent  for  a  common-service  system,  dod’s  prior  experience  with 
the  Pioneer  system  demonstrated  the  difficulties  in  adapting  a  uav  for 
naval  use  and  the  need  for  timely  testing. 

Other  shortcomings  in  the  planned  Short-Range  system  test  program 
were  identified  by  us  and  the  Director,  Defense  Operational  Test  and 
Evaluation.  When  these  shortcomings  were  brought  to  dod’s  attention,  it 
agreed  to  correct  them. 


Recommendations 


dod’s  Short-Range  t'AV  acquisition  strategy  includes  preproduction  oper¬ 
ational  testing,  and  dod’s  initially  procured  systems  represent  a  signifi¬ 
cant  portion  of  the  total  program  quantity  and  are  to  be  deployed  to 
operational  forces.  We  therefore  recommend  that  the  Secretary  of 
Defense  require  that  operational  testing  of  the  Short-Range  iav  be  con¬ 
ducted  in  diverse,  realistic  environments  to  provide  reasonable  assur¬ 
ance  that  it  will  meet  requirements  before  permitting  limited  production 
of  the  land-based  i  av  system.  We  also  recommend  that  the  Secretary' 
limit  Short-Range  i  av  system  production  until  satisfactory  performance 
of  the  Navy  variant  is  demonstrated  and  assure  that  actions  are  taken  to 
correct  the  other  shortcomings  noted  in  the  planned  test  program. 
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Matter  for 

Congressional 

Consideration 


Agency  Comments  and 
Our  Evaluation 


Congress  may  wish  to  consider  whether  the  congressional  intent  for  a 
joint  service  system  is  jeopardized  by  the  substantial  risk  that  the  Navy 
variant  will  be  unsuitable,  and,  if  so,  Congress  may  wish  to  prohibit  full- 
rate  production  of  the  Short-Range  cav  until  performance  of  the  Navy 
variant  is  proven  to  be  satisfactory. 


dod  agreed  or  partially  agreed  with  most  of  the  findings  in  this  report 
but  disagreed  with  the  recommendations  and  matter  for  congressional 
consideration. 

dod  stated  that  an  adequate  evaluation  of  the  system’s  operational 
effectiveness  and  suitability  can  be  accomplished  without  testing  in  all 
environments  in  which  the  system  may  be  employed,  dod  also  stated 
that  the  system’s  acquisition  strategy  and  test  program  are  consistent 
with  the  applicable  dod  directive  and  that  the  system  will  be  tested  in 
representative  operational  environments. 

dod  indicated  that  the  acquisition  risks  are  inherently  low  because  of  the 
nonaevelopmental  nature  of  the  program,  tod  also  stated  that  the  risk 
would  be  further  reduced  by  the  test  program  for  the  Army  and  Marine 
Corps  version,  by  engineering  evaluations  and  design  reviews  of  the 
Navy  variant,  and  by  preproduction  testing  of  the  Navy  variant. 

Our  basic  concern  is  that  the  primary  test  site  does  not  resemble  the 
primary  locations  where  the  system  is  to  be  operationally  deployed 
beginning  in  1994  and  therefore  the  test  results  could  be  very  mis¬ 
leading.  In  fact,  dod’s  restructured  test  plan  recognizes  that  the  testing 
environment  constitutes  a  test  limitation. 

Our  past  work  has  often  demonstrated  the  importance  of  testing  in  a 
realistic  operational  environment,  and  dod’s  acquisition  directive 
requires  that  when  operational  testing  is  conducted,  it  be  conducted 
under  realistic  conditions,  ixjd's  comments  present  no  evidence  that 
negates  the  need  to  realistically  test  the  system  before  committing  to 
production. 

ix)o  also  stated  that  it  has  no  intention  of  delaying  production  of  the 
system  for  the  Army’s  and  Marine  Corps'  use  pending  outcome  of  the 
Navy  variant’s  development.  Finally,  ix>d  pointed  out  that  operational 
testing  of  the  first  Navy  variant  procured  and  other  measures,  such  as 
design  reviews,  would  sufficiently  minimize  the  risk  associated  with  the 
current  acquisition  strategy. 
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Under  the  current  acquisition  strategy  the  scheduled  testing  of  the  Navy 
variant  will  only  reduce  the  risk  of  prematurely  committing  to  produc¬ 
tion  of  that  variant,  rather  than  the  basic  system  which  will  by  then 
already  be  in  full-rate  production.  If  the  Navy  variant  were  to  prove 
unsuccessful  in  the  later  testing,  dod  would  find  itself  fully  committed  to 
production  of  a  system  not  meeting  congressional  intent  for  a  common 
uav.  Experience  with  Pioneer  shows  that  adapting  a  i'av  for  naval  opera¬ 
tions  is  not  a  low-risk  effort  even  though  the  Navy  categorized  it  as  such 
when  justifying  the  program.  We  therefore  continue  to  believe  that  dod 
should  retain  the  Short-Range  system  in  low-rate  production  until  opera¬ 
tional  testing  shows  that  the  Navy  variant's  performance  is  satisfactory. 

dod  refers  to  the  Short-Range  uav  as  a  nondevelopmental  program.  The 
Pioneer  was  also  a  so-called  nondevelopmental  program  in  its  early 
phases  but  later  required  substantial  development  to  solve  problems 
associated  with  naval  operations.  In  addition,  testing  of  the  Army  and 
Marine  Corps  version  of  the  Short-Range  uav  will  not  reduce  risk  associ¬ 
ated  with  the  Navy  variant  because  the  Army  and  Marine  Corps  version 
will  not  incorporate  any  of  the  modifications  necessary  for  operating  in 
a  naval  environment.  Engineering  evaluations,  design  reviews,  and 
preproduction  testing  of  the  Navy  variant  should  help  preclude  a  pre¬ 
mature  commitment  to  production  of  that  variant.  However,  these 
efforts  will  not  reduce  the  risk  of  becoming  prematurely  committed  to 
the  Short-Range  program  as  a  whole  because  the  Army  and  Marine 
Corps  land-based  version  will  be  in  full-rate  production  by  the  time 
these  efforts  are  completed. 


Appendix  I  discusses  the  Short-Range  uav  acquisition  strategy  in  more 
detail.  Appendix  II  describes  our  objective,  scope,  and  methodology. 
Appendix  III  sets  forth  dod  comments. 

Copies  of  this  report  are  being  sent  to  the  Secretaries  of  Defense,  the 
Army,  and  the  Navy;  the  Director  of  the  Office  of  Management  and 
Budget;  selected  congressional  committees;  and  other  interested  parties. 
Copies  will  be  made  available  to  others  upon  request. 


Page  4 


GAO/NSLAD-90-234  Unmanned  Aerial  Vehicles 


B-229489 


This  report  was  prepared  under  the  direction  of  Louis  J.  Rodrigues, 
Director,  Command,  Control,  Communications,  and  Intelligence  Issues, 
who  may  be  reached  on  (202)  275-4841  if  you  or  your  staff  have  any 
questions  concerning  the  report.  Other  major  contributors  to  this  report 
are  listed  in  appendix  IV. 

Sincerely  yours, 


Frank  C.  Conahan 
Assistant  Comptroller  General 
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Evaluation  of  DOD’s  Acquisition  Strategy  for 
the  Short-Range  UAV 


Background 


cays  are  pilotless  aircraft  resembling  small  airplanes  or  helicopters  that 
are  remotely  controlled  or  preprogrammed  to  be  controlled  by  on-board 
equipment,  non  is  acquiring  cays  to  meet  a  variety  of  military  needs. 

A  t 'ay  system  typically  includes  one  or  more  air  vehicles,  a  launch  and 
recovery  system,  and  a  ground  station  for  controlling  the  cay's  night 
and  processing  information  from  the  cay.  The  air  vehicle  carries  the 
system’s  payload,  such  as  a  television  camera,  with  the  specific  type 
depending  on  the  military  mission  to  be  accomplished. 

As  part  of  its  joint  cav  program,  non  is  acquiring  the  Short-Range  system 
for  use  by  the  Army,  Navy,  and  Marine  Corps.  The  system  is  to  provide 
the  capability  to  perform  reconnaissance,  surveillance  of  enemy  activi¬ 
ties,  target  acquisition,  and  other  military  missions. 


Initiation  of  the  Short 
Range  Program 


The  joint-service  Short-Range  program  was  init  iated  in  response  to  con¬ 
gressional  concern  about  the  proliferation  of  cay  programs  and  the  need 
to  acquire  cays  that  could  meet  the  requirements  of  more  than  one  ser¬ 
vice  and  eliminate  duplicative  progiams.  To  achieve  commonality 
among  the  services'  programs,  Congress  eliminated  fiscal  year  1988 
funding  within  the  services’  separate  research,  development,  test,  and 
evaluation  accounts  for  individual  cay  programs  and  consolidated  the 
funding  in  a  joint -service  program. 

Also  during  the  fiscal  year  1988  budget  process,  Congress  refused  to 
authorize  funding  for  the  procurement  of  separate  Navy  and  Army 
Short-Range  cays.  Congress  provided  that  funds  were  available  only  for 
a  joint  program. 


Short-Range  UAV  Program 
Description 


non  initiated  acquisition  of  the  Short-Range  cay  in  fiscal  year  1989  as  a 
common-service  system.  Contracts  were  awarded  in  September  1989  to 
two  firms  for  nondevelopmental1  candidate  systems  for  a  competitive 
fly-off  in  fiscal  year  1991 .  non  plans  to  evaluate  the  candidate  systems 
and  select  the  winning  firm  to  begin  low  rate  production  in  fiscal  year 
1992.  The  initial  procurement  is  to  include  eight  systems  for  deployment 
in  1994. 


1  Nnndi’V 'Hi>|>nii'iital  item  inrans  any  item  I  li;il  is  ( I  1 1  nmnirri  nlly  available  i I  in  use  In  a  I  S 
ag  •iicy  or  foreign  government  wit h  which  flu*  1  'ruled  States  has  a  mutual  defense  coojierat ion  agree  - 
inn,*  or  Ulj  any  of  the  items  in  t  I  i  or  t  L l )  that  require  only  minor  modification 
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The  initially  produced  systems,  designated  as  Block  0,  are  planned  for 
use  by  the  Army  and  Marine  Corps.  However,  pod  does  not  expect  these 
systems  to  fully  meet  performance  requirements  and  thus  plans  a 
research  and  development  program  to  improve  the  Block  0  system's 
capability  and  provide  a  variant  of  the  system  for  naval  use.  The  Navy 
variant  is  designated  as  Block  1,  while  the  improved  Short-Range  system 
is  designated  as  Block  II.  Block  II  improvements  will  include  use  of  a 
diesel  or  jet  fuel  engine  to  eliminate  the  use  of  more  volatile  gasoline. 
The  improvements  will  also  include  growth  in  a  number  of  mission 
payload  and  ground  control  station  capabilities  such  as  automatic 
tracking  of  targets  and  automated  searching  of  designated  areas. 

IK)P  plans  to  procure  a  total  of  53  Short-Range  systems:  27  systems  for 
the  Army,  18  for  the  Marine  Corps,  5  for  the  Navy,  and  3  for  training 
purposes.  The  systems  include  424  air  vehicles  or  8  vehicles  per  system. 
In  addition  to  air  vehicles,  each  system  is  to  consist  of  multiple  pay- 
loads,  a  launch  and  recovery  station,  and  related  equipment.  The  Short- 
Range  t'AV  Decision  Coordinating  Paper,  dated  August  1989,  projected 
total  acquisition  costs  to  be  about  $1.5  billion.  However,  according  to 
pod,  that  amount  may  need  some  revision,  depending  upon  the  winning 
contractor. 


Production  Before 
Realistic  Operational 
Testing 


Despite  the  importance  of  realistic  operational  testing,  the  preproduc¬ 
tion  operational  testing  of  the  Short-Range  system  that  pop  plans  to  con¬ 
duct  will  not  take  place  in  environments  representative  of  where  it  is 
supposed  to  be  deployed.  We  recognize  that  if  the  current  Middle  Hast 
situation  continues  until  1994,  when  the  system  is  to  be  fielded,  the 
Short-Range  system  deployment  plans  could  be  changed.  Nonetheless, 
the  test  results,  which  are  the  basis  for  beginning  production,  may  not 
be  a  valid  indicator  of  the  system's  currently  required  capability. 


The  Navy’s  experience  with  its  Pioneer  Unmanned  Aerial  Vehicle,  first 
procured  in  198U,  illustrates  the  need  for  realistic  operational  testing  of 
the  Short-Range  r.w  system.  Some  of  the  lessons  to  be  learned  from  pro¬ 
duction  before  testing  in  a  realistic  environment  were  vividly  demon¬ 
strated  during  the  acquisition  of  the  Pioneer  system. 


Experience  With  Pioneer  Tin*  predecessors  to  Pioneer  had  been  successfully  used  by  Israeli  forces 

in  the  Middle  Hast.  Thus,  the  Navy  procured  the  Pioneer  as  a 
nondevelopmcntal  system,  without  testing  it.  and  deployed  the  system 
with  its  operational  forces.  Numerous  problems  ensued. 
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•  After  deployment  at  sea,  engine  failures  caused  crashes  and  led  to  tem¬ 
porary  termination  of  Pioneer  operations  pending  engine  modifications 
and  a  change  in  the  type  of  fuel  used.  However,  the  fuel  change  to  avia¬ 
tion  gasoline  then  forced  the  Navy  to  start  a  development  program  for  a 
new  Pioneer  engine  that  would  use  a  less  volatile  fuel  considered  accept¬ 
able  for  shipboard  operations. 

•  Difficulty  in  landing  Pioneer  on  the  ship  also  caused  crashes  and 
required  vehicle  modifications  and  development  of  new  flight  control 
software.  The  air  vehicle  was  also  retrofitted  with  new  foam-filled 
wings  so  that  crashed  vehicles  could  float  until  salvaged. 

•  Problems  in  controlling  Pioneer  at  sea  were  compounded  by  electromag¬ 
netic  interference  from  the  ship’s  high  frequency  radio,  leading  to  at 
least  one  air  vehicle  loss.  This  required  that  the  radio  be  shut  down 
during  Pioneer  operations  pending  modifications  to  harden  Pioneer 
against  the  interTerence. 

•  Placement  of  the  air  vehicle  tracking  unit  behind  the  ship's  superstruc¬ 
ture  created  a  blindspot  where  Pioneer  contact  was  lost.  This  required 
that  the  ship  be  maneuvered  extensively  to  keep  the  air  vehicle  out  of 
the  blindspot  until  another  tracking  unit  could  be  acquired  and  placed  in 
front  of  the  ship’s  superstructure. 

•  Pioneer’s  cumbersome  net  recovery  system  interfered  with  helicopter 
operations  and  sometimes  caused  air  vehicle  and  propeller  damage  when 
the  air  vehicle  landed  in  the  net.  This  required  the  Navy  to  develop  a 
new  net  recovery  system. 

Thus,  the  lack  of  realistic  operational  testing  of  the  Pioneer  cav  system 
left  the  Navy  to  costly  and  time-consuming  trial  and  error  as  it  tried  to 
adapt  a  system  used  in  a  land-based  desert  environment  to  shipboard 
use.  Four  years  after  initial  procurement,  the  Navy  is  still  buying 
replacement  hardware,  such  as  completely  modified  air  vehicles,  to 
bring  Pioneer  systems  up  to  a  minimum  essential  level  of  performance. 


Planned  Short-Range 
Testing  Is  Not  Realistic 


non  picas  to  do  operational  testing  of  the  Short-Range  cav  system  before 
low  rate  production  to  minimize  some  of  the  risk  that  was  inherent  in 
the  Pioneer  cav  acquisition  strategy.  However,  the  planned  testing, 
based  on  current  deployment  plans,  is  not  realistic;  significant  risks 
remain.  Thus,  the  test  results,  which  are  to  be  the  basis  for  beginning 
production,  may  not  be  a  valid  indicator  of  the  system's  capability. 

Operational  testing  is  the  primary  means  of  predicting  weapon  system 
performance  in  a  combat  representative  environment.  Thus,  operational 
testing  should  be  held  in  a  realistic  environment  representing  combat 
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Preproduction  Desert  Testing 


Preproduction  Maritime  Testing 


conditions  to  the  extent  practical.  Our  past  work  has  also  consistently 
illustrated  the  importance  of  conducting  operational  testing  in  a  realistic 
environment.  We  reported  in  December  1 980,-  for  example,  that  results 
of  testing  in  an  unrealistic  environment  can  be  misleading  and  of  only 
limited  usefulness  in  evaluating  system  performance. 

Realistic  operational  testing  of  the  Short-Range  iav  would  require, 
among  other  things,  that  the  system  be  tested  at  locations  resembling 
those  where  it  might  be  used  in  combat.  The  Short-Range  system  is  to  be 
deployed  in  areas  marked  by  hilly  or  mountainous  terrain  with  dense 
forests  and  other  vegetation  and  by  varied  climatic  conditions  such  as 
cloudy  weather,  rain,  snow,  and  other  factors.  Accordingly,  the  Short- 
Range  system  is  supposed  to  be  capable  of  performing  reconnaissance, 
surveillance,  and  target  acquisition  under  a  variety  of  environmental 
conditions. 

dod  plans  to  conduct  preproduction  operational  testing  of  the  Block  0 
system’s  ability  to  perform  such  missions  at  Fort  Huachuca,  Arizona. 
The  operational  test  facility  at  Fort  Huachuca  is  typical  desert  terrain, 
generally  flat  and  lacking  trees  and  other  vegetation.  Our  viewing  of  the 
planned  test  site  showed  few  areas  where  potential  targets  could  be  situ¬ 
ated  other  than  in  open  view.  Thus,  the  planned  testing  to  evaluate  the 
Short-Range  system's  capability  to  locate  targets  under  various  environ¬ 
mental  conditions  in  which  the  EAV  is  to  operate  may  prove  misleading. 

dod  also  plans  to  conduct  preproduction  maritime  testing  at  a  second 
location,  the  Pacific  Missile  Test  Center.  However,  this  testing  will  not 
include  operations  from  Navy  ships.  Instead,  testing  will  include  opera¬ 
tions  from  runways  on  land  and  will  therefore  not  demonstrate  the 
system's  capability  to  operate  in  a  naval  environment. 

As  discussed  earlier,  the  Navy’s  Pioneer  experience  demonstrated  the 
difficulties  in  adapting  a  i  av  system  for  use  at  sea.  After  crashing  14  of 
the  air  vehicles  during  maritime  operations  and  21  overall,  and  encoun¬ 
tering  numerous  other  performance  problems,  the  Navy  redesigned  and 
modified  practically  the  entire  Pioneer  system.  Despite  those  changes, 
made  at  a  cost  of  about  $50  million  in  Research  N  Development  and 
replacement  hardware.  Pioneer  does  not  meet  its  intended  performance 
requirements. 
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Evaluation  of  DOD’s  Acquisition  Strategy  for 
the  Short-Range  UAV 


Congressional  Intent 
for  a  Joint-Service 
System  Is  at  Risk 


Some  Shortcomings  of 
Original  Test  Plans 
Addressed 


dod  plans  to  commit  to  full-rate  production  of  the  Short-Range  i  'av 
system  before  verifying  that  it  can  be  modified  to  meet  Navy  require¬ 
ments.  dod’s  acquisition  policies  require  that  operational  testing  be  com¬ 
pleted  prior  to  full-ra^e  production.  Hence,  dod  plans  to  conduct 
operational  testing  in  1992  before  the  system’s  full-rate  production 
begins.  However,  the  system  to  be  tested  is  the  Block  0  version  for  use 
by  the  Army  and  Marine  Corps  and  will  not  incorporate  those  features 
necessary  to  operate  from  Navy  ships. 

Congress  has  already  refused  to  authorize  funding  for  separate  Army 
and  Navy  short-range  systems.  If  subsequent  operational  testing  of  the 
Navy  variant  were  to  show  that  a  modified  Short-Range  system  is 
unsuitable  for  Navy  use,  dod  would  be  faced  with  the  alternatives  of  (1) 
cancelling  the  Short-Range  production  and  starting  a  new  program  to 
acquire  a  system  meeting  both  Army  and  Navy  needs  or  (2)  continuing 
the  Short-Range  production  for  the  Army  and  Marine  Corps  and 
acquiring  a  separate  system  for  the  Navy. 


We  and  officials  in  the  Office  of  the  Secretary  of  Defense  identified 
other  shortcomings  in  the  Short-Range  system’s  original  test  program. 
Subsequent  to  completion  of  our  work,  however,  the  uav  Joint  Program 
Office  agreed  to  address  these  concerns  by  significantly  revising  its  test 
plans. 

Initially,  we  were  concerned  that  the  iow-rate  initial  production  decision 
for  the  Short-Range  uav  was  planned  to  follow  an  early  operational 
assessment  period  rather  than  the  currently  planned  operational  testing. 
An  official  from  the  Office  of  the  Secretary  of  Defense  stated  that  an 
early  operational  assessment  should  not  be  considered  a  substitute  for 
operational  testing.  Although  dod  policy  allows  the  initiation  of  low  rate 
production  based  on  an  early  operational  assessment,  dod  revised  its 
acquisition  strategy  to  provide  for  preproduction  operational  testing. 

Another  shortcoming  was  that  the  test  plan  contained  a  provision  that 
none  of  the  criteria  forjudging  the  Short-Range  system's  performance 
was  to  be  considered  as  absolute  pass  or  fail  measures.  Our  concern  with 
this  provision  was  that  it  conflicted  with  the  Secretary  of  Defense’s 
intent  as  recently  stated  in  the  July  1989  Defense  Management  Report  to 
the  President.  The  Secretary  stated  that  dod  policy  will  be  to  discipline 
the  acquisition  process  by  defining  minimum  required  accomplishments 
for  advancing  from  one  phase  of  the  acquisition  process  to  the  next. 

This  provision  was  removed  from  the  test  plan. 
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Wo  and  Defense  officials  had  concerns  about  the  success  threshold  for 
locating  targets  accurately  during  Short-Range  testing.  The  percentage 
of  targets  to  be  located  accurately  during  testing  had  been  tentatively 
set  at  one-fifth  of  the  percentage  specified  in  the  Short-Range  uav  per¬ 
formance  requirements.  In  response  to  these  concerns  the  Joint  Program 
Office  is  revising  its  test  plans  to  increase  the  goal  for  locational  accu¬ 
racy  so  that  it  corresponds  to  the  performance  requirements. 
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Objective,  Scope,  and  Methodology 


In  response  to  a  request  from  the  CL. m  man,  House  Armed  Services 
Committee,  we  reviewed  Don's  acquisition  strategy  fov  the  Short-Range 
day.  The  Chairman  expressed  concern  about  the  lack  of  success  on  past 
i  av  programs  and  the  need  to  assure  that  weaknesses  contributing  to 
the  lack  of  success  do  not  recur. 

We  obtained  the  information  for  this  report  by  reviewing  records  and 
interviewing  officials  of  the  i  ay  Joint  Program  Office  and  Office  of  the 
Secretary  of  Defense  in  Washington,  I).C.;  Marine  Corps  officials  in 
Quantico,  Virginia;  Short-Range  day  Project  Office  officials  at  Redstone 
Arsenal,  Huntsville,  Alabama;  and  Army  Training  and  Doctrine  Com¬ 
mand  officials  at  Fort  Huachuca,  Arizona.  We  also  visited  the  planned 
day  test  site  at  the  day  Joint  Training  Center  at  Fort  Huachuca. 

We  compared  *■’  acquisition  strategy  of  the  Short-Range  day  and  the 
Pioneer  day,  a  _  viously  procured  system  that  encountered  problems 

when  deployed  with  the  Navy’s  operational  forces.  After  our  initial 
efforts  indicated  weaknesses  in  the  planned  Short-Range  system  testing, 
we  concentrated  on  evaluating  the  planned  test  program  and  comparing 
it  to  the  Pioneer  experience.  V,\_  obtained  information  on  Pioneer  from 
officials  of  the  Naval  Air  Systems  Command.  Washington,  D.C. 

To  determine  the  extent  to  which  performance  will  be  demonstrated,  we 
compared  the  planned  testing  for  the  Short-Range  system  to  its  perform¬ 
ance  requirements  documents.  We  also  analyzed  planned  testing  docu¬ 
ments  to  determine  w  hether  they  were  consistent  with  dod's  testing 
policies. 

Our  review  was  performed  from  October  1989  through  April  1990  in 
accordance  with  generally  accepted  government  auditing  standards,  dod 
provided  written  comments  on  a  draft  o'  this  report.  These  comments 
are  presented  and  evaluated  in  appendix  III. 
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Note  GAO  comments 
supplementing  those  in  the 
report  text  appear  at  the 
end  of  this  appendix 


See  comment  1 


DIRECTOR  OF  DEFENSE  RESEARCH  AND  ENGINEERING 
WASHINGTON,  DC  20301-3010 


August  20,  1990 


Hr.  Frank  C.  Conahan 
Assistant  Comptroller  General 
National  Security  and  International 
Affairs  Division 
General  Accounting  Office 
Washington,  DC  20548 

Dear  Mr.  Conahan: 

This  is  the  Department  of  Defense  (DoD)  response  to  the 
General  Accounting  Office  (GAO)  Draft  Report,  "UNMANNED  AERIAL 
VEHICLE:  Realistic  Testing  Needed  Before  Production  of  Short- 

Range  System,"  dated  10  July  1990, (GAO  Code  395120/OSD  Case 
8410) . 

The  DoD  concurs  or  partially  concurs  with  most  of  the 
findings  in  the  GAO  report.  However,  the  DoD  does  not  concur 
with  either  recommendation  or  the  matter  for  congressional 
consideration . 

The  Test  and  Evaluation  Master  Plan  for  the  Unmarned  Aeriai 
Vehicle-Short  Range  system,  which  describes  the  test  program,  is 
being  restructured  and  will  be  reviewed  by  the  Office  of  the 
Secretary  of  Defense  (OSD).  The  responsible  OSD  officials  wi i i 
approve  the  test  program  only  if  it  is  adequate  and  consistent 
with  applicable  statutes  and  DoD  directives.  Therefore,  the  GAO 
assertion  of  inadequate  testing  cannot  be  addressed  at  this 
time . 


Detailed  comments  on  the  report  findings,  recommendations, 
and  matter  for  congressional  considerations  are  provided  in  the 
enclosure.  Thank  you  for  this  opportunity  to  review  and  to 
comment  on  the  draft  report. 

Sincerely, 


~c\. 

harles  M.  Herzfeld 


Enclosure 
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GAO  DRAFT  REPORT  -  DATED  JULY  10,  1990 
(GAO  CODE  395120)  OSD  CASE  8410 

"UNMANNED  AERIAL  VEHICLE:  REALISTIC  TESTING  NEEDED  BEFORE 
PRODUCTION  OF  SHORT  RANGE  SYSTEM" 

DEPARTMENT  OF  DEFENSE  COMMENTS 

***** 


FINDINGS 


FINDING  A:  Initiation  of  the  Short-Range  Unmanned  Aerial 
Vehicle  Program.  The  GAO  reported  that  a  joint  service  program 
was  initiated  by  the  Congress  in  response  to  concern  about  the 
proliferation  of  unmanned  aerial  vehicle  programs  and  the  need 
to  acquire  vehicles  that  meet  the  requirements  of  more  than  one 
Service.  The  GAO  explained  that,  in  order  to  achieve 
commonality,  Congress  eliminated  the  FY  1988  funding  within  the 
Services'  separate  research,  development,  test,  and  evaluation 
accounts  for  individual  programs  and  consolidated  the  funding  in 
a  joint-service  program.  The  GAO  also  reported  that,  during  FY 
1988,  the  Congress  refused  to  authorize  funding  for  the 
procurement  of  separate  Navy  and  Army  unmanned  aerial  vehicles-- 
providing  funds,  instead,  for  a  joint  program.  The  GAO  reported 
that  contracts  were  awarded  in  September  1989  to  two  firms  for 
non-developmental  candidate  systems  for  a  competitive  fly-off  in 
FY  1991.  The  GAO  further  reported  that  the  DoD  plans  to  procure 
a  total  of  53  Short-Range  systems:  27  for  the  Army;  18  for  the 
Marine  Corps;  5  for  the  Navy;  3  for  training  purposes.  The  GAO 
found  that  the  Short-Range  Unmanned  Aerial  Vehicle  Decision 
Coordinating  Paper,  dated  August  1989,  projected  total 
acquisition  costs  of  about  $1.5  billion.  The  GAO  noted  that 
program  officials  now  indicate  that  that  amount  may  be  revised, 
(pp.  7-10/GAO  Draft  Report) 

DOD  RESPONSE:  Concur .  Depending  on  the  winning  contractor,  the 
actual  program  amount  may  need  slight  revision. 


FINDING  B:  Production  Before  Realistic  Operational  Testing . 

The  GAO  reported  that,  contrary  to  its  stated  acquisition 
principles,  the  DoD  does  not  plan  to  conduct  preproduction 
operational  testing  of  the  Short-Range  system  in  environments 
typical  of  where  it  may  be  used  in  combat.  The  GAO  concluded, 
therefore,  that  the  test  results  (from  those  tests  that  will  be 
conducted)  may  not  be  a  valid  indicator  of  the  system's 
capability.  The  GAO  pointed  out  that  the  Navy's  experience  with 
the  PIONEER  Unmanned  Aeriai  Vehicle,  illustrated  the  need  for 
realistic  operational  testing  of  the  Short-Range  Unmanned  Aerial 
Vehicle  system.  The  GAO  noted  that  the  lack  of  realistic 
operational  testing  of  the  PIONEER  Unmanned  Aerial  Vehicle 
system  led  the  Navy  to  costly  and  time  consuming  trial  and 
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\0v\  on  pp  9-10 


See  comment  2 


error,  as  it  tried  to  adapt,  for  shipboard  uses,  a  system  used 
in  a  land-based  desert  environment.  The  GAO  reported  that  the 
Navy  is  still  buying  replacement  hardware  to  bring  the  PIONEER 
systems  up  to  a  minimum  essential  level  of  performance,  (pp.  10- 
12/GAO  Draft  Report) 

DoD  RESPONSE:  Nonconcur .  The  Unmanned  Aerial  Vehicle-Short 
Range  test  program  must  be  viewed  in  the  context  of  its 
acquisition  strategy.  The  acquisition  strategy  is  non- 
developmental .  Production  funds  are  used  to  procure  systems  for 
a  competitive  fly-off  and  for  follow-on  systems.  Full-scale 
development  funds  are  not  used  to  procure  any  systems. 

Production  of  two  systems  from  each  of  two  contractors  was 
approved  at  the  Milestone  II/IIIA  decision  point.  The  Milestone 
IIIB  decision  point,  which  constitutes  the  low-rate  production 
decision,  will  address  the  selection  of  one  contractor  to 
produce  up  to  eight  systems.  The  Milestone  IIIC  decision  point, 
which  constitutes  the  full  production  decision,  will  address  the 
additional  procurement  of  up  to  43  additional  Short  Range 
systems . 

As  part  of  selection  criteria  and  prior  to  exercise  of  the  first 
low  rate  production  option,  two  tests  will  be  performed.  The 
first  test,  Technical  Evaluation  Test,  will  test  each  contractor 
system  against  the  required  system  technical  performance  and 
includes  survivability  testing  using  the  threat  arrays  at  the 
Naval  Weapons  Center. 

The  second  test,  Operational  Test  IIA,  will  assess  each 
contractor  system  for  operational  effectiveness  and  suitability 
when  employed  by  representative  user  personnel  under  realistic 
operational  conditions.  Operational  Test  IIA  will  also  include 
a  survivability  evaluation  using  the  results  obtained  from  the 
Technical  Evaluation  Test,  which  will  have  used  the  threat 
arrays  at  the  Naval  Weapons  Center.  Thus,  Operational  Test  IIA 
is  a  two-month  test  program  that  incorporates  field  exercises 
under  simulated  combat  conditions,  in  which  the  system  will  be 
operated  by  soldiers  and  marines.  Training  will  be  provided  to 
bring  user  personnel  to  operating  proficiency  prior  to 
commencement  of  Operational  Test  IIA.  The  test  results  from  the 
Technical  Evaluation  Test  will  be  utilized  to  evaluate  each 
system’s  performance,  and  results  from  Operational  Test  IIA  will 
be  utilized  to  evaluate  each  system's  operational  utility.  That 
information  will  be  provided  for  source  selection  evaluation. 

Prior  to  a  Milestone  IIIC  decision  to  enter  full  rate 
production.  Operational  Test  IIB  will  be  performed  on  the 
winning  contractor's  system.  Operational  Test  IIB  will  be 
conducted  utilizing  representative  user  personnel  under 
realistic  operational  conditions  to  evaluate  operational 
effectiveness  and  suitability  to  include  reliability, 
maintainability,  ef f ectiveness-of-training ,  logistic 
suppor tabi 1 i ty ,  and  any  correction  of  deficiencies  previously 
noted.  Operational  Test  IIB  is  also  a  two  month  test  program  in 
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which  the  system  will  be  operated  by  Army  and  Marine  Corps 
personnel  in  a  field  environment  that  incorporates  threat 
simulators . 

See  comment  3  Therefore,  performance  requirements  and  operational 

effectiveness  and  suitability  will  be  assessed  prior  to  the 
Milestone  IIIB  decision  to  enter  low  rate  production,  and  prior 
to  a  Milestone  IIIC  decision  to  enter  full  scale  production.  It 
should  be  noted  that  OSD  disapproved  the  initial  UAV-SR  Test  and 
Evaluation  Master  Plan  ,  which  outlines  the  UAV-SR  test  program, 
in  December  1989.  The  primary  reasons  for  the  disapproval 
concerned  inadequacies  in  the  operational  test  program.  When  a 
revised  Master  Plan  is  reviewed,  the  responsible  OSD  officials 
will  approve  the  operational  test  program  only  if  it  is  adequate 
and  consistent  with  applicable  statutes  and  DOD  directives. 

The  DoD  recognizes  significant  problems  were  encountered  in  the 
effort  to  adapt  the  PIONEER  for  use  by  the  Navy.  Based  on 
lessons  learned  from  that  process,  the  DoD  is  using  the  block 
upgrade  concept  to  develop  the  Navy  variant  of  the  Short  Range 
Unmanned  Aerial  Vehicle. 

See  comment  4  Concerning  statements  on  the  PIONEER,  the  Navy's  baseline 

concept  of  acquisition  initially  provided  three  PIONEER  systems 
that  were  used  during  an  extensive  operational  assessment 
period,  during  which  time  the  requirements  for  United  States 
Marine  Corp  (USMC)  land  and  United  States  Navy  (USN)  maritime 
operations  were  further  defined.  Rather  than  a  "time  consuming 
trial  and  error"  period,  this  assessment  was  conducted  by  both 
the  operational  users  under  actual  operational  conditions  and  by 
personnel  at  the  Pacific  Missile  Test  Center.  The  resulting 
assessment  provided  Navy  planners  with  the  necessary  data  to 
effectively  make  a  baseline  decision  had  the  PIONEER  program 
continued  as  planned.  Additionally,  the  parts  being  bought  are 
spares  necessary  to  maintain  the  PIONEER  at  its  currently 
acceptable  operational  level  of  performance. 

FINDING  C;  Planned  Short-Range  Testing  Is  Not  Realistic.  fhe 
GAO  concluded  that,  while  the  DoD  plans  to  do  some  preproduction 
operational  testing  of  the  Short-Range  Unmanned  Aerial  Vehicle 
system,  the  planned  testing  is  not  realistic  and  significant 
risks  remain.  The  GAO  further  concluded  that  any  test  results, 
which  are  to  be  the  basis  for  beginning  production,  may  not  be  a 
valid  indicator  of  the  system's  capability. 

The  GAO  reported  that  realistic  operational  testing  of  the 
Short-Range  Unmanned  Aerial  Vehicle  would  require  that  the 
system  be  tested  at  locations  resembling  those  where  it  might  be 
used  in  combat.  The  GAO  concluded  that--because  the  system, 
which  is  to  be  deployed  in  diverse  environments,  including  hilly 
or  mountainous  terrain,  dense  forests,  rain,  snow,  and  cloudy 
weather,  is  only  going  to  be  tested  at  Fort  Huachuca,  Arizona, 
in  desert  terrain — the  results  may  be  misleading. 
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Now  on  pp  10- r 


I 


See  comment  5 


See  comment  6 


The  GAO  also  found  that  the  DoD  plans  to  conduct  preproduction 
maritime  testing  at  the  Pacific  Missile  Test  Center.  The  GAO 
further  found,  however,  that  testing  will  not  include  operations 
from  Navy  ships.  The  GAO  reported  that  the  planned  testing  will 
only  include  operations  from  runways  on  land  and,  therefore,  not 
demonstrate  the  system's  capability  to  operate  in  a  naval 
environment.  The  GAO  referred  to  the  difficulties  in  adapting 
the  PIONEER  to  use  at  sea.  According  to  the  GAO,  (1)  17  of  the 
original  40  vehicles  crashed,  (2)  the  Navy  redesigned  and 
modified  nearly  the  entire  system  at  a  cost  of  about  $50  million 
in  research  and  development  and  replacement  hardware,  and  (3) 
the  system  still  does  not  meet  its  intended  performance 
requirements.  (pp.  12-15/GAO  Draft  Report) 

DoD  RESPONSE:  Partially  concur.  The  test  program,  composed  zf 

both  technical  and  operational  testing,  will  be  conducted  in  the 
severe  flight  environments  of  Fort  Huachuca  (high  altitude,  high 
temperature,  dust,  etc.)  and  the  Pacific  Missile  Test  Center 
(moisture/water,  fog  and  salt/fog,  etc.).  In  order  to  increase 
the  realism  of  operational  tests,  camouflage  will  be  uspd  to 
mask  targets.  The  Pacific  Missile  Test  Center  early  chamber 
technical  testing  will  expand/supplement  the  natural  conditions 
encountered  to  cover  the  remaining  environmental  specification 
requirements.  The  Naval  Air  Propulsion  Center  engine  technical 
tests  will  determine  the  capability  of  the  engines  to  meet  all 
altitude  and  temperature  specifications. 

An  overriding  factor  in  the  identification  of  Fort  Huachuca, 
Arizona,  as  the  site  for  the  OT-IIA  was  the  extensive  airspace 
and  range  scheduling  limitations  presented  by  other  potential 
test  sites.  The  environment  at  Tort  Huachuca  is  extremely 
severe  on  system  operation.  The  combination  of  altitude  and 
temperature  is  equivalent  to  operating  at  an  altitude  of  10,000 
feet.  The  terrain  at  Fort  Huachuca  includes  mountains  and  some 
treed  areas.  The  testing  will  take  place  during  the  Fort 
Huachuca  rainy  season.  During  this  period,  low-hanging  clouds 
and  periods  of  intense  rain  can  be  expected.  The  rainy  season 
begins  in  the  June  timeframe.  The  Unmanned  Aerial  Vehicle  Joint 
Project  Office  is  in  the  process  of  identifying  an  additional 
site,  with  environmental  conditions  other  than  those  of  Fort 
Huachuca,  for  conduct  of  a  portion  of  the  Operational  Test  IIB. 

To  correct  the  record,  maritime  operations  training  and  testing 
of  the  PIONEER  Unmanned  Aerial  Vehicle  has  resulted  in  fourteen 
air  vehicles  lost  rather  than  the  seventeen  cited  in  this 
finding.  As  previously  stated,  the  DoD  recognizes  significant 
problems  were  encountered  in  the  effort  to  adapt  the  PIONEER  for 
use  by  the  Navy  on  board  ships.  The  Department's  block  upgrade 
concept  to  develop  the  Navy  variant  of  the  Short  Range  Unmanned 
Aerial  Vehicle  is  designed  to  take  advantage  of  the  lessons 
learned  on  PIONEER. 

Plans  for  the  operational  testing  of  the  Navy  Block  I  variant  of 
the  Unmanned  Aerial  Vehicle-Short  Range  during  the  follow-on 
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operational  test  and  evaluation  phase  have  not  yet  been 
developed.  However,  the  DoD,  particularly,  operational  test 
i  agencies,  recognize  the  need  to  test  weapon  systems  onboard 

ships,  if  the  systems  will  be  used  in  that  environment. 

FINDING  D :  Congressional  Intent  For  A  Joint-Service  System  Is 
At  Risk.  The  GAO  observed  that  DoD  acquisition  policies  require 
j  that  operational  testing  be  completed  prior  to  full-rate 

production.  The  GAO  found,  however,  that  the  DoD  plans  to 
commit  to  full-rate  production  of  the  Short-Range  Unmanned 
Aerial  Vehicle  before  verifying  that  it  can  be  modified  to  meet 
Navy  requirements.  The  GAO  reported  that  the  DoD  plans  for 
|  operational  testing  in  1992--before  fuli-rate  production 

beg ins--i nvol ve  the  Block  0  version  for  use  by  the  Army  and 
j  Marine  Corps,  but  will  not  incorporate  those  features  necessary 

to  operate  from  Navy  ships.  The  GAO  concluded,  that  if 
i  subsequent  operational  testing  of  the  Navy  variant  were  to  show 

that  a  modified  Short-Range  system  is  unsuitable  for  Navy  use, 

I  the  DoD  would  be  faced  with  the  alternatives  of  (1)  canceling 

the  Short-Range  production  and  starting  a  new  program  to  acquire 
i  a  system  meeting  both  Army  and  Navy  needs  or  (2)  continuing  the 

Short-Range  production  for  the  Army  and  Marine  Corps  and 
Now  on  p  12  1  acquiring  a  separate  system  for  the  Navy.  (pp.  i5-16/GAO  Draft 

[  Report) 

J  DoD  RESPONSE:  Partially  concur.  As  provided  for  in  the  current 

contracts,  as  part  of  the  downselect  process,  each  contractor 
will  submit  for  evaluation  an  engineering  study,  which  will 
detail  system  configuration  changes  required  for  shipboard 
See  comment  7  operations.  Subsequent  to  the  downselect,  the  Navy  shipboard 

j  variant  system  will  then  complete  technical  and  detailed  design 

j  reviews.  Full-rate  production  of  Block  0  is  planned  to  occur 

following  design  reviews  of  the  Navy  shipDoard  variant. 

[  Operational  Testing  of  the  Block  I  system  will  be  conducted 

onboard  ships  prior  to  exercising  the  production  option  of  the 
|  naval  variant. 

During  an  Early  Operational  Assessment,  the  Navy  will  assess  the 
See  comment  8  !  Block  0  Unmanned  Aerial  Vehicle-Short  Range  system's  capaDility 

to  meet  shipboard  requirements.  Of  primary  interest  is  the 
I  capability  of  the  Short-Range  Block  0  system  to  operate  from 

ships  while  performing  Reconnaissance,  Surveillance,  and  Target 
Acquisition  missions. 

It  is  important  to  realize  that  the  preponderance  of  the  current 
requirement  for  the  Unmanned  Aerial  Vehicle-Short  Range  is  for 
See  comment  9  the  Block  0  version.  Only  five  of  the  53  vehicles  planned  are 

the  Navy  variant.  Block  I.  The  approach  recommended  by  the  GAO 
would  delay  fielding  of  the  48  Block  0  systems  by  one  to  two 
years,  while  the  Block  I  Navy  variant  completed  its  operational 
test  ing . 

!  The  best  way  to  operate  an  Unmanned  Aerial  Vehicle  from  ships, 

particularly  launch  and  recovery  operations,  is  still  being 
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explored.  It  is  possible  tnat  the  Block  0  air  vehicle  may  not 
be  usable  or  may  require  extensive  modifications  for  shipboard 
use.  However,  this  would  not  preclude  system  interoperability 
or  commonality  in  the  use  of  other  Unmanned  Aerial  Vehicle-Short 
Range  system  elements  (mission  planning  stations,  ground  control 
stations,  data  terminals,  etc,)  in  the  Block  I  system.  Testing 
of  the  Block  0  system  will  provide  information  applicable  to  the 
Navy  variant.  That  information  will  be  available  before  the 
full-rate  production  decision  for  the  Block  0  system. 

FINDING  E:  Some  shortcomings  of  the  The  Original  Test  Plans 
Addressed .  The  GAO  reported  that  it  (along  with  officials  in 
the  Office  of  the  Secretary  of  Defense)  identified  other 
shortcomings  in  the  Short-Range  system's  original  test  program 
that  the  Unmanned  Aerial  Vehicle  Joint  Program  Office  agreed  to 
address  by  significantly  revising  the  test  plan.  The  GAO 
reported  that,  initially,  the  low-rate  initial  production 
decision  for  the  Short-Range  Unmanned  Aerial  Vehicle  was  planned 
to  follow  an  early  operational  assessment  period,  rather  than 
the  currently  planned  operational  testing.  The  GAO  noted  tha£, 
subsequently,  the  DoD  revised  its  acquisition  strategy  to 
provide  for  preproduction  operational  testing. 

The  GAO  reported  that  another  shortcoming  was  that  the  test  plan 
contained  a  provision  that  none  of  the  criteria  for  judging  the 
Short-Range  system's  performance  was  to  be  considered  as 
absolute  pass  or  fail.  The  GAO  found  this  provision  in  conflict 
with  the  intent  of  the  Secretary  of  Defense  to  discipline  the 
acquisition  process  by  defining  minimum  required  accomplishments 
for  advancing  from  one  phase  to  the  next,  as  recently  stated  in 
July  1989  Defense  Management  Report  to  the  President.  The  GAO 
reported  that,  subsequently,  the  conflicting  provision  was 
removed  from  the  test  plan. 

The  GAO  also  reported  that  both  GAO  and  DoD  officials  had 
concerns  about  the  success  threshold  for  locating  targets 
accurately  during  Short-Range  testing.  The  GAO  found  that  the 
percentage  of  targets  to  be  located  accurately  during  testing 
had  been  tentatively  set  at  one-fifth  of  the  percentage 
specified  in  the  Short-Range  Unmanned  Aerial  Vehicle  performance 
requirements.  The  GAO  reported  that,  in  response  to  those 
concerns,  the  Joint  Program  Office  is  revising  its  test  plans  to 
increase  the  goal  for  locational  accuracy  so  that  it  corresponds 
to  the  performance  requirements.  (pp.  16-18/GAO  Draft  Report) 

DoD  RESPONSE:  Concur .  The  revised  Unmanned  Aerial  Vehicle- 
Short  Range  Test  and  Evaluation  Master  Plan  has  been  revised  to 
better  reflect  operational  requirements  and  to  provide  for 
adequate  operational  testing  prior  to  Milestone  IIIB  and  lie 
decisions . 


*  •  •  *  * 
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See  comment  10 


Now  on  p  2 


See  comment  1 1 


RECOMMENDATIONS 

RECOMMENDATION  1:  The  GAO  recommended  that  the  Secretary  of 
Defense  require  that  the  Short-Range  Unmanned  Aerial  Vehicle 
system  be  operationally  tested  in  diverse,  realistic 
environments  to  provide  reasonable  assurance  that  it  will  meet 
requirements  before  permitting  limited  production  of  the  land- 
based  system. 

DoD  RESPONSE  Nonconcur .  Department  of  Defense  Directive  5000.3 
states  that  demonstration  of  a  system's  technical  capabilities 
and  its  operational  effectiveness  and  suitability  will  be  a  key 
requirement  for  decisions  to  commit  significant  resources,  to 
advance  from  one  acquisition  phase  to  another,  and  to  field  the 
system.  An  adequate  evaluation  of  a  system's  operational 
effectiveness  and  suitability  can  be  accomplished  without 
testing  in  all  of  the  operational  environments  in  which  a  system 
may  be  employed.  The  Unmanned  Aerial  Vehicle  Short  Range 
acquisition  strategy  and,  in  particular,  the  test  program,  is 
consistent  with  the  DoD  directive  for  a  non-developmental  item 
program. 

The  planned  Technical  Evaluation  Test  and  Operational  Test  IIA 
will  evaluate  the  system's  performance  and  operational 
effectiveness/suitability.  The  planned  test  program  will  test 
the  system  in  meaningful  representative  operational  environments 
and  will  provide  data  to  support  a  Milestone  IIIB  decision  to 
enter  low  rate  production. 

RECOMMENDATION  2:  The  GAO  recommended  that  the  Secretary  of 
Defense  limit  Short-Range  Unmanned  Aerial  Vehicle  system 
production  until  satisfactory  performance  of  the  Navy  variant  is 
demonstrated  and  assure  that  actions  are  taken  to  correct  the 
other  shortcomings  noted  m  the  planned  test  program,  (pp.  3- 
4/GAO  Draft  Report) 

DoD  RESPONSE:  Nonconcur .  The  Unmanned  Aerial  Vehicle-Short 
Range  acquisition  strategy,  with  its  provisions  for  production 
lot  options,  limits  production  of  the  Block  I  system  until  its 
performance  is  demonstrated.  However,  the  Department  of  Defense 
does  not  intend  to  delay  production  of  a  successful  Block  0 
system,  pending  the  outcome  of  the  Block  I  development.  As 
previously  noted,  48  of  the  53  systems  planned  are  the  Block  0 
version.  The  Unmanned  Aerial  Vehicle  -  Short  Range  acquisition 
strategy  includes  a  block  upgrade  approach,  which  allows  for  the 
evaluation  and  fielding  of  an  initial  baseline  configuration, 
followed  by  block  upgrades  to  meet  the  full  operational 
requirements.  The  modular  architecture  of  the  system  will 
facilitate  upgrades.  Block  0  is  the  initial,  baseline  system. 
Block  I  is  the  Navy  shipboard  variant.  The  described 
acquisition  strategy  has  been  fully  reviewed  and  approved,  and 
is  in  keeping  with  the  non-developmental  item  nature  of  the 
system.  The  cognizant  congressional  committees  have  also  been 
appraised  of  the  acquisition  strategy  being  approved. 
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As  provided  in  the  current  contracts,  each  contractor  will 
submit  for  evaluation  as  part  of  the  downselect  process,  an 
engineerinn  study  which  will  detail  system  configuration  changes 
required  for  shipboard  operations.  These  studies  will  be 
completed  this  fiscal  year.  Following  this,  a  FY1992  contract 
option  may  be  exercised  for  the  integration,  demonstration,  and 
test  of  the  Block  I  Navy  shipboard  variant.  Not  until  FY1994 
will  the  government  exercise  a  contract  option  which  will 
include  the  purchase  of  one  system  with  Block  I  upgrades. 

See  comment  7  Operational  testing  of  this  system  in  a  shipboard  environment 

will  be  conducted  prior  to  exercising  an  option  to  procure 
additional  Navy  systems.  This  is  the  first  of  five  Navy 
shipboard  systems  to  be  purchased.  Therefore,  the  Block  I 
engineering  studies  provided  to  each  competing  contractor  as 
part  of  the  downselect  process;  the  technical  design  and  the 
detailed  design  review  subsequent  to  down  select;  and  the 
operational  testing  of  the  initial  Block  I  system  prior  to 
procuring  additional  systems,  will  sufficiently  minimize  the 
risk  associated  with  the  block  upgrade  approach. 


FOR  CONSIDERATION  BY  THE  HOUSE 
COMMITTEE  ON  ARMED  SERVICES 


\0.v  on  p  3 


See  comments  2  4  9 
and  1 1 


MATTER  FOR  CONSIDERATION ;  To  assure  that  congressional  intent 
for  a  common-service  system  is  achieved,  the  GAO  suggested  that 
the  Committee  may  wish  to  prohibit  full-rate  production  of  the 
Short-Range  Unmanned  Aerial  Vehicle  until  performance  of  the 
Navy  variant  is  proven  to  be  satisfactory.  (p.  4/GAO  Draft 
Report ) 

DoD  RESPONSE :  Nonconcur .  The  non-developmenta  1  nature  of  the 
Unmanned  Aerial  Vehicle  Short  Range  Program  eliminates  much  of 
the  risk  inherent  in  a  developmental  program.  The  risk  is 
further  minimized  by  the  Block  0  test  program,  the  Block  I 
engineering  evaluations  and  design  reviews,  and  the  pre- 
production  tests  of  the  Block  I  Navy  shipboard  variant.  The 
resultant  risk  is  so  small  that  the  program  delays  and 
additional  costs  inherent  in  this  suggestion  exceed  any 
additional  risk  avoidance. 

Operational  testing  of  the  Unmanned  Aerial  Vehicle-Short  Range 
system  before  procurement  of  the  Navy  Block  I  variant  will 
provide  a  basis  for  Block  I  development.  The  GAO  cited  problems 
with  the  performance  of  the  PIONEER  system  onboard  ships.  Many 
of  those  problems  were  associated  with  the  difficulties  of 
adapting  a  land-based  system  for  shipboard  use.  The  DoD  has 
learned  from  that  experience.  Many  uncertainties  and  unknowns 
are  still  associated  with  the  use  of  Unmanned  Aerial  Vehicle 
systems  in  worldwide  military  operations  across  a  wide  spectrum 
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of  combat  intensity, 
measured  approach  to 


The  Block  upgrade  concept  will  provide  a 
resolving  those  uncertainties  and  unknowns. 
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The  following  are  cap's  comments  on  the  Department  of  Defense's  letter 
dated  August  20,  1990.  and  its  accompanying  enclosure. 


1.  We  believe  that  concerns  about  inadequate  testing  need  to  be 
addressed  as  part  of  the  ongoing  restructuring  of  the  Test  and  Evalua¬ 
tion  Master  Plan. 

2.  The  nondevelopmental  nature  of  the  program,  does  not  negate  the 
requirement  for  realistic  testing,  non  directive  5000.3  on  testing  states 
that  nondevelopmental  items  shall  be  guided  by  the  same  non  acquisi¬ 
tion  principles  as  other  systems.  This  directive  requires  testing  of  suita¬ 
bility  for  use  in  combat  in  an  environment  as  operationally  realistic  as 
possible. 

pop’s  comments  do  not  address  our  main  concern  that  the  Short-Range 
system’s  preproduction  operational  testing  will  be  limited  primarily  to 
the  desert  environment  which  does  not  resemble  the  primary  locations 
where  the  system  is  to  be  deployed.  These  locations  are  marked  by 
mountainous  terrain  with  dense  forests  and  adverse  climatic  conditions. 
From  that  standpoint,  the  planned  testing  of  the  Short-Range  system  is 
unrealistic  and  could  yield  misleading  results  on  the  system's  capability 
to  locate  targets. 

It  is  also  important  to  point  out  that  pop’s  comments  address  the 
planned  testing  prior  to  the  full-rate  production  decision,  while  our 
report  addresses  the  lack  of  realistic  testing  in  pod’s  preproduction  test 
plan.  Moreover,  the  site  referred  to  by  ixip  as  Operational  Test  HR  has 
not  yet  been  identified  by  pop.  Therefore,  pop’s  response  indicating  that 
the  operational  tests  leading  up  to  a  full-rate  production  decision  will  be 
conducted  under  realistic  operational  conditions  is  premature.  Program 
officials  indicated  that  discussions  have  occurred  within  pop  about 
selecting  a  site  that  would  be  more  representative  of  a  planned  deploy¬ 
ment  location  than  Fort  Huachuca.  but  no  decision  has  been  made. 

3.  Our  report  recognizes  that  operational  testing  of  the  Short-Range 
system  will  be  held  before  the  low-rate  and  full-rate  production  deci¬ 
sions.  Our  concern  with  the  testing  before  low-rate  production  (opera¬ 
tional  test  1 1 A )  is  that  it  is  to  be  restricted  primarily  to  an  unrealistic 
environment.  Our  concern  with  the  testing  before  full-rate  production  is 
that  it  will  not  evaluate  the  system’s  capability  to  operate  in  a  naval 
environment.  Testing  of  the  Navy  variant  is  to  occur  only  after  pop  has 
committed  to  full-rate  production. 
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4.  We  cited  the  Pioneer's  problems,  in  part,  to  illustrate  the  need  for 
realistic  operational  testing  before  committing  to  production  In  our 
view,  ;x)D  is  not  capitalizing  on  what  should  be  a  lesson  learned  from  the 
Pioneer  experience. 


Pioneer  problem,  ',  were  discovered  by  deploying  the  system  and  waiting 
to  see  if  it  worked.  When  problems  occurred,  the  Navy  was  forced  to 
continually  bring  in  experts  from  the  Naval  Air  Propulsion  Center,  the 
Naval  Air  Test  Center,  the  Pacific  Missile  Test  Center,  or  send  personnel 
to  the  subcontractors  in  Israel  to  find  solutions.  In  1987,  the  Navy  pro¬ 
duced  an  Integrated  Action  Plan  to  address  the  numerous  shortcomings 
of  the  system.  In  this  p'an  the  Pioneer  acquisition  strategy  was  entitled 
“Operate.  Learn,  Fix,  Operate."  We  see  no  difference  between  this  and 
"trial  and  error.” 


The  Navy  recently  (1990)  described  the  Pioneer  system's  level  of  per¬ 
formance  as  a  “minim-  -sscntial  capability  "  (underscoring  added). 

We  would  point  <.  ,  that  .jOD  and  the  Navy  expended  $28  million  in 
rosear 'h  -u.u  development  funds  and  an  additional  $22  million  in  pro¬ 
em  vment  for  replacement  hardware  over  4  years  to  reach  this  level  of 
;  “rformance. 

5.  Although  Fort  Huachuea  has  a  high  altitude,  the  terrain  is  generally 
flat  and  treeless.  Moreover,  there  is  no  indication  in  the  revised  draft 
test  plan  that  pod  intends  to  overcome  this  drawback. 

To  determine  whether  the  rainy  season  at  Fort  Huachuea  was  sigi  L 
cant  and  might  indicate  something  about  the  ability  of  the  Short-Range 
system's  ability  to  operate  in  general  conditions  of  wetness,  we  acquired 
recent  meteorological  data  for  the  closest  metropolitan  area  to  the  Fort 
Huachuea  test  site.  It  showed  that  ( 1 )  less  than  3  inches  of  rain  fell  per 
month  during  the  3-month  rainy  season  (June,  July,  and  August)  and 
(2)  rain  fell  in  measurable  amounts  on  only  n  days.  This  environment  is 
not  similar  to  that  of  the  primary  locations  where  the  system  is  to  be 
deployed.  For  example,  at  one  planned  deployment  location  the  rainy 
season  is  7  months,  and  measurable  amounts  of  rain  occur  on  average  Id 
days  per  month.  Average  monthly  accumulation  exceeds  8  inches  per 
month,  more  than  Fort  Huachuea  can  expect  in  its  entire  3-month  rainy 
season. 

Range  scheduling  limitations  presented  by  other  potential  test  sites,  as 
identified  in  ixm’s  response,  should  not  in  our  view  be  an  overriding 
factor.  The  need  to  perform  realistic  testing  should  be  the  deciding 
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factor.  Although  l>oi>  may  he  looking  for  a  test  site  ot Iter  than  Fort 
Iluachuea.  the  <  Iperational  Test  IIH  will  not  occur  until  after  production 
of  the  Short-Ranee  system  has  started. 

h.  In  a  draft  of  t  his  report .  we  identified  1 7  Pioneer  air  vehicles  as  being 
lost  during  niaritinv  operations.  The  final  report  has  been  modified  to 
reflect  mop's  position  that,  of  the  21  vehicles  lost.  14  were  lost  during 
maritime  operations.  (  Also  see  comment  4.) 

7.  We  do  not  disagree  with  hop's  comments;  however,  the  fact  remains 
that  the  Short-Range  system  will  he  in  full-rate  production  before  any 
operational  test >”g  of  the  Navy  variant  is  done. 

S.  According  to  the  Director.  Defense  Operational  Test  and  Evaluation, 
an  early  operational  assessment  is  no  substitute  for  operational  testing. 

!*.  We  did  not  recommend  delaying  the  fielding  of  the  Block  0  system. 

Our  recommendation  would  require  that  the  Block  0  system  be  retained 
in  low-rate  production  pending  verification  that  the  Block  1  variant  will 
meet  Navy  needs.  This  would  reduce  the  risk  of  tx)n  becoming  fully  com¬ 
mitted  to  production  of  a  system  not  meeting  common-service  require¬ 
ments  Further,  pop's  plan  is  to  retrofit  Block  0  systems  to  incorporate 
design  features  of  the  upgraded  configurations;  therefore,  limiting  Block 
0  production  would  reduce  the  risks  associated  with  the  retrofit. 

Testing  of  the  land-based  system  ( Block  0)  will  not  provide  reasonable 
assurance  that  the  various  elements  will  meet  the  Navy's  requirements. 
Such  assurances  can  only  he  obtained  through  operationally  testing  the 
system  in  a  realistic  naval  environment. 

10.  pop  cannot  conclusively  determine  whether  the  Short-Range  PAY  will 
be  operationally  effective  or  suitable  based  on  its  planned  testing.  The 
operational  environment  in  which  pop  intends  to  test  is  not  at  all  similar 
to  the  diverse  environments  of  the  primary  locations  where  ix>n  intends 
to  deploy  the  system.  In  fact  pop's  draft  restructured  test  plan  recog¬ 
nizes  that  the  testing  environment  constitute.-,  a  test  limitation. 

1  1.  Our  recommendation  does  not  affect  low-rate  production  of  the  land- 
hased  system.  Although  specifically  proscribed  during  congressional 
consideration  of  fiscal  year  DISH  pay  funding,  ixtp's  adherence  to  the 
cm  rent  Short-Range  r.w  acquisition  strategy  could  result  in  separate 
Army  and  Navy  Short-Range  i  ay  systems  if  the  Navy  variant  proves  a 
failure.  This  would  not  be  consistent  with  congressional  direction. 
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Appendix  III 

Comments  From  the  Department  of  Defense 
and  Our  Evaluation 


Don's  acquisition  strategy  calls  for  entering  full-rate  production  of  the 
Short-Range  system  for  use  by  the  Army  and  Marine  Corps  before  con¬ 
ducting  any  testing  of  the  block  upgraded  Navy  variant.  Scheduled 
testing  of  the  Navy  variant  will  only  reduce  the  risk  of  prematurely 
committing  to  production  of  that  system,  not  the  basic  system,  which 
will  already  be  in  full-rate  production.  If  the  Navy  variant  were  to  prove 
unsuccessful  in  the  later  testing,  DODwould  find  itself  fully  committed  to 
production  of  a  system  not  meeting  congressional  intent  for  a  common 
l  AV.  Further,  as  brought  out  in  our  report,  experience  with  Pioneer 
shows  that  adapting  a  cav  for  naval  operations  is  not  a  low-risk  effort 
even  though  the  Navy  categorized  it  as  such  when  justifying  the  pro¬ 
gram.  We  therefore  continue  to  believe  that  non  should  retain  the  Short- 
Range  system  in  low-rate  production  until  operational  testing  shows 
that  the  Navy  variant’s  performance  is  satisfactory. 
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Major  Contributors  to  This  Report 


National  Security  and 
International  Affairs 
Division,  Washington, 
D.C. 


Charles  A.  Ward,  Evaluator-in-Charge 


Atlanta  Regional 
Office 


.Jackie  B.  Guin,  Assistant  Director 
Bobby  D,  Hall,  Evaluator 
Terry  D.  Wyatt,  Evaluator 
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